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Abstract. Multi-agent planning (MAP) approaches are typically oriented at solving 
loosely-coupled problems, being ineffective to deal with more complex, strongly-related 
problems. In most cases, agents work under complete information, building complete 
knowledge bases. The present article introduces a general-purpose MAP framework de¬ 
signed to tackle problems of any coupling levels under incomplete information. Agents 
in our MAP model are partially unaware of the information managed by the rest of 
agents and share only the critical information that affects other agents, thus maintain¬ 
ing a distributed vision of the task. 

Agents solve MAP tasks through the adoption of an iterative refinement planning 
procedure that uses single-agent planning technology. In particular, agents will devise 
refinements through the Partial-Order Planning paradigm, a flexible framework to build 
refinement plans leaving unsolved details that will be gradually completed by means 
of new refinements. Our proposal is supported with the implementation of a fully- 
operative MAP system and we show various experiments when running our system 
over different types of MAP problems, from the most strongly-related to the most 
loosely-coupled. 

Keywords: Planning & scheduling; Multi-agent systems 


1. Introduction 

Planning is the art of building control algorithms that synthesize a course of 
action to achieve a desired set of goals from an initial situation. Traditionally, 
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planning has been regarded as a centralized process in which a single entity is in 
charge of devising a plan that satisfies the problem goals. 

Multi-Agent Planning (MAP) generalizes the problem of planning in domains 
where several agents plan and act together. MAP introduces a social approach 
to planning (Nguyen and Katarzyniak, 2009), focusing on the collective effort 
of multiple planning entities to accomplish tasks by combining their knowledge, 
information and capabilities. This is required when agents are unable to solve 
their tasks by themselves, or at least can accomplish them better (more quickly, 
completely, precisely, or certainly) when working with others (Durfee, 2001). 

MAP is concerned with planning by multiple agents, i.e., distributed plan¬ 
ning, and planning for multiple agents, i.e., planning for multi-agent execution, 
thus giving rise to a great variety of tools and techniques. The approach tradition¬ 
ally adopted by the Multi-Agent Systems (MAS) research community assumes 
that, in general, agents are self-interested and that there is not a common goal to 
solve, thus focusing on coordinating the activities of multiple agents in a shared 
environment (desJardins, Durfee, Ortiz and Wolverton, 1999). In agent-oriented 
approaches, the ultimate objective is to ensure that the agents’ local objectives 
(private goals) will be achieved by their plans and so the emphasis is put on 
distributed execution, plan synchronization and collaborative activity at run¬ 
time planning (Durfee and Lesser, 1991; Tambe, 1997; Kaminka, Pynadath and 
Tambe, 2002). All in all, these techniques use planning as a means to controlling 
and coordinating agents rather than building a competent and joint plan, and 
so they are very appropriate for the design of real-time systems (Micacchi and 
Cohen, 2008). 

In planning-oriented approaches dealing with contexts in which agents are 
assumed to be cooperative, the objective is to study how planning can be ex¬ 
tended into a distributed environment or, more particularly, on the construction 
of a competent plan by several planning entities. There exist different approaches 
to address this objective, varying according to the typology of the planning prob¬ 
lem to solve. In particular, the adoption of one or another strategy depends on 
the coordination needs of the problem, i.e., to which extent agents are able to 
make their own plans without affecting what the other agents are planning to 
do. Thus, when agents are assumed to be relatively independent, they carry out 
their planning activities individually and exchange information about their local 
plans, which they iteratively refine and revise until they fit together in order 
to ensure that the resulting plan will jointly execute in a coherent and efficient 
manner (desJardins et al., 1999). This has been the predominant approach in 
cooperative MAP, existing a large body of research on post-planning coordina¬ 
tion, i.e., solving inconsistencies among local plans that have been constructed 
separately. The well-known Partial Global Planning (PGP) framework (Durfee 
and Lesser, 1991) is one of the first techniques that allows agents to communicate 
and merge their local plans. Ever since, many works on plan merging methods 
for building a joint plan given the local plans of each participating agent have 
arisen (see section for a detailed description). 

The application of MAP to loosely-coupled multi-agent tasks, in which agents 
have little interaction to each other, is still an active area of research. Some re¬ 
cent works in this line, where agents are engaged in some cooperative behaviour, 
have emerged lately. These works follow a common approach that consists of 
coordinating the local solutions developed by the agents. For instance, the work 
in (Kvarnstrom, 2011) considers that agents have sequential threads of execu¬ 
tion and interactions only occur when distributing sub-plans to individual agents 
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for plan execution. This approximation follows a single-agent planning and dis¬ 
tributed coordination. The work in (Brafman and Domshlak, 2008) applies indi¬ 
vidual planning and coordinates the local solutions through the resolution of a 
Constraint Satisfaction Problem (CSP). In an extension of this latter work, au¬ 
thors use a distributed CSP to solve inconsistencies among agents’ plans (Nissim, 
Brafman and Domshlak, 2010). 

Most of the aforementioned approaches turn out to be inefficient at the time 
of solving strongly-related problems in which the number of coordination points 
among agents is large (Nissim et ah, 2010). To deal with these problems, other 
MAP models use a unified approach in which planning and coordination of activ¬ 
ities are integrated rather than being treated as independent processes (Jonsson 
and Rovatsos, 2011; Belesiotis, Rovatsos and Rahwan, 2010). However, these ap¬ 
proaches do not achieve high performance in loosely-coupled problems because 
the reasoning procedures rely very strongly on a high degree of interdependency 
between the agents’ actions. 

The problem of building a competent joint plan among several planning enti¬ 
ties has been generally dismissed by the MAS community, more concerned with 
the development of coordination mechanisms for agents, and ignored by the 
planning community, which has traditionally resorted to efficient single-agent 
algorithms to solve planning problems. MAP is not only about a divide-and- 
conquer strategy to tackle large planning problems, it is also about the devel¬ 
opment of techniques for planning entities that are geographically or spatially 
distributed. While one might expect the number of coordination points in in¬ 
herently distributed problems not to be very large, another issue that comes up 
is the distribution of information among agents. In frameworks like those pre¬ 
sented in (Brenner and Nebel, 2009; Belesiotis et ah, 2010) agents communicate 
all the available information and build complete knowledge bases, i.e., agents 
have complete information on the MAP task. However, in large-size problems 
with heterogeneous agents, building complete knowledge bases is not viable. Be¬ 
sides efficiency issues, agents may be unable to manage the information handled 
by other agents as they may have different knowledge and abilities. 

In this paper, we present a novel approach to cooperative MAP that allows to 
efficiently solve problems with any level of interaction among agents. Unlike other 
techniques, our MAP system is capable of solving from the most loosely-coupled 
problems to the most strongly-related problems. The key point to address this 
aspect is to use a refinement planning approach (Kambhampati, 1997) that allows 
agents to interleave planning and coordination, or more specifically, to coordinate 
their plans during planning. We also allow heterogeneous agents to work under 
incomplete information, sharing only the critical information that affects other 
agents and maintaining a distributed vision of the MAP task. This issue, which 
has been ignored in almost all of the MAP approaches, is of key importance 
to efficiently handle inherently distributed problems. Last but not least, our 
MAP approach is entirely based on the use of single-agent planning technology 
adapted to a multi-agent context. More precisely, agents follow the Partial-Order 
Planning paradigm (Nguyen and Kambhampati, 2001; Younes and Simmons, 
2003). 

As well as introducing the MAP architecture and a theoretical model for 
multi-agent planning, our proposal is supported with the implementation of a 
fully-operative MAP system. The empirical evaluation of the system demon¬ 
strates this novel approach to be effective when dealing with both strongly- 
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related problems and loosely-coupled problems in which agents manage incom¬ 
plete information. 

This paper is organized as follows: section [^summarizes some background on 
the main topics related to this work and reviews the most recent literature on 
MAP; section introduces the example MAP scenario we will use to illustrate 
the different aspects of our framework; section [^ outlines our MAP architec¬ 
ture; section [^ presents the theoretical planning model upon which our system 
is based; section outlines the planning language used to model MAP tasks; 
section provides an overview of the MAP algorithm followed by the agents; 
section [^describes the first stage of our MAP algorithm, the initial information 
exchange; section outlines second stage of the MAP algorithm, the refinement 
planning and coordination protocol; section[^presents the experimental results, 
and finally, section El concludes and summarizes our future lines of research. 

2. Background 

Our MAP model builds upon several single-agent planning techniques. This sec¬ 
tion provides a review on the principal single-agent planning concepts used in 
our MAP approach as well as the most relevant and recent approaches to co¬ 
operative MAP. We also outline the most relevant works on MAP architectures 
and frameworks and we conclude by summarizing the main contributions and 
novelties of our approach. 


2.1. Single-agent planning 

Single-agent planning is regarded as a search process by which a single entity 
synthesizes a set of actions (plan) to reach a set of objectives from an initial sit¬ 
uation (Weld, 1999). Over the last years, single-agent planning has experienced 
great advances, specifically in the construction of domain-independent heuristics. 
Nowadays, it is possible to find a great variety of planning systems. The most 
recent planners combine different techniques in order to increase the algorithms 
efficiency: landmarks (Richter and Westphal, 2010), domain transition graphs 
(Helmert, 2006), forward-chaining partial-order planning (Coles, Coles, Fox and 
Long, 2010), probes (Lipovetzky and Geffner, 2011) or divide-and-conquer strate¬ 
gies (Dreo, Saveant, Schoenauer and Vidal, 2011), among others. 

The work in (Blum and Furst, 1997) introduced the concept of Relaxed Plan¬ 
ning Graph, which has proven to be one of the most effective constructs to devise 
heuristics in state-space planning (Hoffmann and Nebel, 2001). This technique 
has been integrated in many single-agent planning frameworks and has also been 
extended to a distributed context (Zhang, Nguyen and Kowalczyk, 2007). 

While state-space planners such as Fast Forward (Hoffmann and Nebel, 2001) 
are still a relevant research topic, plan-space planning has been replaced by other 
more efficient techniques. However, plan-space planning has recently seen a re¬ 
vival since its flexibility makes it specially suitable for distributed environments. 

Among plan-space search algorithms, the Partial-Order Planning (POP) ap¬ 
proach (Penberthy and Weld, 1992; Younes and Simmons, 2003) is particularly 
relevant. POP performs a plan-based, backward search process, refining partial 
plans through the addition of actions, causal links and ordering constraints. POP 
is based on the least commitment strategy (Weld, 1994), which defers planning 
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decisions during the search process and introduces partial-order relations among 
actions rather than enforcing a concrete order among them. The particular na¬ 
ture of the POP paradigm (absence of states, backward search) makes it difficult 
to devise competitive heuristics to guide the search process. Although some re¬ 
cent works reformulate the basic algorithm to improve its performance (Coles 
et ah, 2010), POP has been discontinued by the planning community in favor of 
other approaches. Nevertheless, it is still used in temporal planning and MAP 
environments as it is a flexible paradigm to handle concurrency (Boutilier and 
Brafman, 2001). 


2.2. Cooperative Multi-Agent Planning 

MAP extends the single-agent planning problem by distributing the planning 
task among several entities which work together to devise a competent joint 
plan that meets the problem goals. This generalization entails some differences 
to the more restrictive single-agent planning approach. MAP can be viewed as 
the problem of coordinating agents in a shared environment where information 
is distributed (desJardins et ah, 1999). This definition emphasizes two aspects 
of MAP that are not present in single-agent planning: the coordination of the 
planning activities and the distribution of the information among agents. 

In general, solving a cooperative MAP task involves the following stages 
(Durfee, 2001): 1) global goal refinement, 2) task allocation, 3) coordination be¬ 
fore planning, 4) individual planning, 5) coordination after planning, and 6) plan 
execution. Some of the previous stages can be avoided or combined. For instance, 
some works do not distribute the goals explicitly (avoiding stage 2) (Belesiotis 
et ah, 2010; Brenner and Nebel, 2009), while others apply only coordination after 
planning (avoiding stage 3) (Van Der Krogt and De Weerdt, 2005; Cox, Durfee 
and Bartold, 2005). 

MAP problems can be classified according to their coupling level, a measure of 
the number of interactions or coordination points among agents that will arise 
during the task resolution (Brafman and Domshlak, 2008). In loosely-coupled 
problems, each problem goal problem is likely to be solved by a single agent, 
while goals in strongly-related problems tend to require the cooperation of several 
agents. The number of coordination points in a MAP problem determines which 
approaches are more suitable to solve it efficiently. 

A wide range of MAP approaches put the emphasis on coordination after 
individual planning (coordination is performed at stage 5 of the MAP scheme 
described above). This way, these frameworks perform the planning and coordi¬ 
nation stages independently and separately, combining or merging solutions into 
a global joint plan (Durfee, 2001; de Weerdt and Clement, 2009; Tonino, Bos, 
de Weerdt and Witteveen, 2002; Kaminka et ah, 2002). 

Different coordination techniques have been proposed for merging and gath¬ 
ering several individual plans into a single joint plan. The Partial Global Plan¬ 
ning framework (Durfee and Lesser, 1991) and its extension, the Generalized 
Partial Global Planning approach (Decker and Lesser, 1992), allow agents to 
communicate their local plans to the rest of agents and then they merge this 
information into their own partial global plan in order to improve it. This it¬ 
erative process goes on until the agents’ local plans fit together. The work in 
(Tonino et ah, 2002) proposes a post-planning coordination approach based on 
the iterative revision of the agents’ local plans. Agents in this model cooperate 
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by mutually adapting their individual plans, with a focus on maximizing their 
common or individual profit. (Nissim et ah, 2010) introduces a cooperative MAP 
approach for loosely-coupled systems in which agents carry out planning indi¬ 
vidually through a state-based planner (Hoffmann and Nebel, 2001; Coles, Fox, 
Long and Smith, 2008). The resulting local plans are then coordinated by solving 
a distributed Constraint Satisfaction Problem. The approach in (Van Der Krogt 
and De Weerdt, 2005) solves inconsistencies among the local plans devised by 
self-interested agents through plan repair. Other proposals deal with insincere 
agents by combining planning, coordination, and execution (Ephrati and Rosen- 
schein, 1996) or consider the communication needs that arise when plans are 
being executed (Tang, Norman and Parsons, 2010). 

The aforementioned plan merging methods follow a common approach: agents 
build plans individually while a subsequent independent process is used to coor¬ 
dinate these plans. This approach is suitable for solving loosely-coupled problems 
efficiently as the agents’ local solutions in these problems present few interdepen¬ 
dencies with each other. Thus, plan merging through post-planning coordination 
is an appropriate method to tackle problems in which agents can solve the differ¬ 
ent problem goals independently and the majority of the environment resources 
are not shared. 

However, plan merging methods present several limitations. On the one hand, 
goals must be a priori allocated to each agent or at least implicitly distributed 
among the planning entities, as agents perform their planning activity in an 
isolated manner. Because of this, methods based on plan merging lose flexi¬ 
bility against other MAP proposals. On the other hand, the previous merging 
approaches have proven to be inefficient when solving strongly-related prob¬ 
lems in which most of the resources are shared and most of the goals require 
cooperation among agents (Nissim et ah, 2010). The individual planning com¬ 
bined with a post-planning coordination strategy is not adequate to solve these 
strongly-related problems, since merging may introduce exponentially many or¬ 
dering constraints in problems which require a coordination effort. 

Another research trend on cooperative MAP stresses the importance of com¬ 
bining and integrating planning and coordination activities, i.e., apply coordina¬ 
tion during planning. Hence, this trend can be seen as an extension of single-agent 
planning to MAP, providing a unified vision of MAP. Proposals in this line focus 
on the cooperative incremental construction of a joint plan, allowing agents to 
perform their planning activity over a centralized plan representation. This is a 
more suitable approach than the plan merging techniques for tackling strongly- 
related MAP problems with a large number of coordination points, as agents 
work over a centralized plan representation and planning and coordination of 
activities are carried out in an integrated way. 

The proposal in (desJardins et al., 1999) applies the continual planning ap¬ 
proach, which interleaves planning and execution and coordinates agents by syn¬ 
chronizing them at execution time (Brenner and Nebel, 2009). The approach in 
(Jonsson and Rovatsos, 2011) introduces the best-response planning algorithm, 
which iteratively improves the quality of the agents’ plans through single-agent 
planning technology. Finally, the works in (Belesiotis et al., 2010; Pajares and 
Onaindia, 2012) solve inconsistencies among agents’ plans through a coordina¬ 
tion protocol based on iterated dialogues. Agents discuss and argument about 
the different plan proposals until the agents’ viewpoints are aligned and an agree¬ 
ment is reached. 

The integrated planning and coordination approach followed by the afore- 
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mentioned MAP models copes with a wider range of MAP problems than the 
plan merging method, which can only deal with simpler, loosely-coupled prob¬ 
lems. In addition, the continual revision and coordination of the agents’ plans 
provides better results in terms of plan quality. However, integrating planning 
and coordination entails higher communication costs for loosely-coupled prob¬ 
lems than using plan merging, as coordination has to be performed throughout 
the planning process, thus introducing an overhead. Hence, the simpler plan 
merging approach is far more effective for small-size and non-complex planning 
tasks. 

Research on cooperative MAP, traditionally carried out by the planning com¬ 
munity, has generally overlooked the management of incomplete information, an 
active research topic, though, within the MAS community. Planning with incom¬ 
plete information has several different meanings: that certain facts of the initial 
state are not known, that operators can have random or nondeterministic effects, 
or that the plans built contain sensing operations and are branching (Haslum 
and Jonsson, 1999). In our case, we interpret incomplete information as agents 
not knowing the initial state completely and being total or partially unaware of 
the information managed by other agents. 

The issue of incomplete information has been treated from two different per¬ 
spectives: the probabilistic way, with the development of formal models such as 
Dec-POMDPs (Decentralized Partial Observable Markov Decision Processes) for 
coordination among multiple agents in contexts with partial observability (Wu, 
Zilberstein and Chen, 2011; Kumar, Zilberstein and Toussaint, 2011); and the 
epistemological way, which assumes that agents have beliefs about the state of the 
world and beliefs over the other agents’ knowledge (Kraus, 1997; Doshi, 2007). 
This latter approach has been widely used in games of incomplete information 
(Gmytrasiewicz and Doshi, 2005). Both perspectives define agents as having an 
imprecise or uncertain view of the world and of the other agents’ information 
but, to the best of our knowledge, there are not proposals to deal with ignorance, 
i.e., local views of agents that reflect agent’s unawareness over the information 
of the rest of agents. This introduces a complexity factor in the planning pro¬ 
cess as agents can only plan on the basis of their information, being ignorant 
on the planning decisions of other agents. It is important to note, though, that 
the information unknown to one agent does not have a direct impact on the 
agents’ choices because its actions are not involved with the unknown piece of 
information. However, this absence of information may have an indirect impact 
in the overall planning process and quality of the plan. 


2.3. Architectures and frameworks for MAP 

The design of architectures and frameworks constitutes another active research 
field in MAP. Over the last years, some relevant works in MAP frameworks have 
been published. The work in (Wilkins and Myers, 1998) presents a complete 
MAP architecture for large-scale problem solving, which organizes agents into 
planning cells committed to a particular planning process. The TAEMS domain- 
independent coordination framework (Lesser, Decker, Wagner, Carver, Garvey, 
Horling, Neiman, Podorozhny, Prasad, Raja et ah, 2004) provides agents with 
planning capabilities, and applies the GPGP approach to coordinate them. 

Other MAP architectures are based on general-purpose MAS platforms, rather 
than being designed from the ground up. MAS platforms, such as Magentix2 
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(Fogues, Alberola, Such, Espinosa and Garcia-Fornes, 2010; Argente, Botti, Car- 
rascosa, Giret, Julian and Rebollo, 2011) or JADE (Bellifemine, Poggi and Ri- 
massa, 2001), provide the sets of services, conventions and knowledge required by 
agents to interact with each other. For instance, the domain-independent multi¬ 
agent system infrastructure RETSINA (Sycara and Pannu, 1998) introduced a 
planning component (Paolucci, Shehory, Sycara, Kalp and Pannu, 2000). Once 
integrated into the agents’ internal architecture, this component provides them 
with planning capabilities. 

Similarly, our MAP approach builds upon the Magentix2 MAS platform, 
which provides the communication services required by the agents. From this 
base, we introduce the additional components to provide the agents with planning 
capabilities and allow them to tackle MAP tasks. 


2.4. Contributions of our model 

Our novel approach to cooperative MAP can be classified into the research trend 
that integrates planning and coordination. The MAP system achieves two main 
objectives: 1) it solves complex strongly-related problems as well as loosely- 
coupled problems without losing generality; and 2) it allows heterogeneous agents 
to work under incomplete information, sharing only the critical information that 
affects other agents and being partially unaware of the other agents’ information 
on the MAP task. 

Our MAP approach focuses on a novel method that combines single-agent 
planning technologies and a refinement-based methodology. More precisely, we 
combine a distributed refinement planning procedure (Kambhampati, 1997) and 
an individual Partial-Order Planning (POP) (Nguyen and Kambhampati, 2001; 
Younes and Simmons, 2003). Agents incrementally build local refinements to 
a certain base plan through their local POPs, and coordinate these partial so¬ 
lutions through the refinement planning process. Empirical evaluation proves 
this method to perform effectively for both strongly-related and loosely-coupled 
problems. 

Another key feature of our method is the ability to work under incomplete 
information. Unlike many MAP proposals, agents in our approach do not re¬ 
quire to build complete knowledge bases, but they can be partially unaware of 
the information on the initial state and the knowledge and abilities of the rest of 
agents. Our PDDLS.l-hased MAP specification language (Kovacs, 2011) defines 
this partial visibility of the agents, allowing to specify which information can be 
shared with other agents for cooperation purposes. Agents exchange the share¬ 
able information with other agents through the construction of a distributed 
Relaxed Planning Graph (Zhang et ah, 2007) and perform planning while be¬ 
ing partially unaware of the other agents’ knowledge. This way, our proposal 
stresses the importance of privacy in a MAP context, as agents share only the 
essential information that affects other agents and are partially unaware of the 
information held by the rest of planning entities. 


3. Motivating example 

This section introduces the example MAP scenario we use in the following to 
illustrate the concepts presented throughout this paper. The example of appli- 
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Fig. 1. Transportation and storage scenario 


cation, depicted in Figure [T] describes a transportation and storage scenario 
in which two agents (Agl and Ag2) take the role of transport agencies and a 
third agent (Ag3) manages a storage facility. Transport agents deliver packages 
through a network of cities. In turn, the warehouse agent is in charge of storing 
and delivering packages to the trucks. Packages can be either raw materials or 
final products. Agents in the MAP task are entrusted with two different goals: 
deliver the final product pi to city cA and the raw material p3 to city cE. 

This scenario includes bidirectional links among cities that allow transport 
agents to move trucks from one city to another. Transport agents Agl and Ag2 
can perform three different actions: they can load and unload packages in the 
trucks and they can move the trucks between cities in their working areas. Agl 
and Ag2 can only move trucks within the cities included in their working areas, 
depicted in Figure as two different circles. This way, transport agents have to 
interact and cooperate in order to deliver packages to a different working area. 

A possible plan to solve the scenario depicted in Figure [^involves Agl loading 
the raw material p3 in the truck tl. Then Agl would handle tl to Ag2 in cB 
or cD, both included in the working areas of Agl and Ag2, and Ag2 would take 
care of transporting the product to cE. This leads to a key aspect of our model: 
in order to promote cooperation, Agl should share with Ag2 the information on 
the position of tl once it reaches cB or cD. As we will discuss in the following 
section, agents will share the information that is relevant for other agents in 
order to successfully cooperate. 

The warehouse agent Ag3 is in charge of interacting with the trucks to store 
raw materials and deliver final products. The warehouse has a table in which 
packages can be stacked and unstacked. Packages are swapped in the city in 
which the warehouse is placed, the exchange city. As seen in Figure cF is the 
exchange city used by Ag2 and Ag3 to swap packages. 

Ag2 and Ag3 will also share information on the packages they leave in the 
exchange city, which will be necessary for them to interact. For example, to 
accomplish the first goal of the task (transporting the final product pi to cA), 
Ag3 will deliver pi to the exchange city cF, informing Ag2 about the position of 
the package. Then, Ag2 will load pi in the truck tl and will drive tl to cB or 
cD. Finally, Agl will perform the final transportation, delivering pi to city cA. 
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Solution plan 



Fig. 2. MAP system architecture 


4. Multi-Agent Planning architecture 

The architecture of our MAP system is depicted in Figure]^ The MAP architec¬ 
ture basically consists of a set of agents endowed with planning capabilities and 
an underlying communication infrastructure that allows them to interact with 
each other. 

All the agents share the same internal structure, and the internal planning 
algorithm followed by each agent is a POP procedure, so they all develop the 
same rationale. However, since agents handle different information and knowl¬ 
edge, that is, incomplete information on the MAP task and different planning 
abilities, our MAP system features heterogeneous agents. In the example of ap¬ 
plication presented in section two agents play the role of transport agencies 
and a third agent manages a storage facility. The first two agents will likely per¬ 
form similar actions like driving vehicles from one location to another, which will 
be different from the planning abilities of the third agent devoted to stack and 
arrange packages in a warehouse. Additionally, agents will have a different view 
of the planning task accordingly to their abilities and initial knowledge; thus, 
the first two agents will have information about the trucks and roads connecting 
the different locations, and the third agent will manage the information about 
the packages and the hoists in the warehouse. 

Together with the planning agents, the MAP architecture provides a set of 
components that allow the user to interact with the platform. The main compo¬ 
nents of the MAP architecture are: 

— Graphical User Interface (GUI): This component allows the user to interact 
with the MAP system. The user requests the resolution of a MAP task by 
providing, for each agent involved in the task, two input files encoded throimh 
our MAP specification language, the domain and problem file (see section^. 
The first file defines the typology and the planning capabilities of the agent. 
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Fig. 3. Internal structure of a planning agent 


while the second file defines the concrete aspects of the task it has to solve. 
Once a solution is found, it is displayed to the user through the GUI. 

— MAP manager: This component interacts with the GUI by collecting the user’s 
request for a plan and assigning the MAP task to a subset of agents that are 
available, i.e., they are not solving any particular planning task at the moment. 
Agents are fully reconfigurable and can be reused when they become available 
again by assigning a new MAP task to them. 

— Pool of planning agents: The architecture includes a pool of planning agents 
which all share the same internal structure shown in Figure Agents are 
configurable through the domain and problem files provided by the user, which 
define the agents’ knowledge and abilities. Once a subset of the agents in the 
pool receive a planning task, they start working together to find a solution 
plan. 

— Gommunication infrastructure: Agents interact with each other through a com¬ 
munication infrastructure, which allows them to exchange messages by follow¬ 
ing the FIPA communication protocols (Kone, Shimazu and Nakajima, 2000). 
The developed MAP system uses the Magentix2 MAS platform (Fogues et ah, 
2010) as its communication infrastructure. 

The internal structure of the planning agents includes several modules to ac¬ 
complish the requirements of our refinement planning approach. Through these 
modules, agents make plan refinements over a base plan, select the best alter¬ 
native from a set of refinement plans and communicate with each other (see 
Figure]^. Although agents have the same internal structure, they have different 
planning abilities and visibility over the MAP task as defined in the domain and 
problem file provided by the user. The internal modules of a planning agent are: 

— Gommunication module: Through this module, each planning agent interacts 
with the rest of agents via the communication infrastructure. The commu¬ 
nication module receives messages from the rest of agents and transmits the 
received information to the rest of internal modules of the planning agent. 
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When the agent wants to communicate with other agents, this module is in 
charge of sending the messages through the communication infrastructure (the 
Magentix2 MAS platform). Hence, this module acts as an interface between 
the planning agent and the rest of agents in the MAP task. 

— Planning module: This module is in charge of performing the actual plan¬ 
ning search. It includes an embedded Partial-Order Planner which has been 
modified to be able to start the planning process from an incomplete plan 
and return valid refinements instead of complete solution plans. The planning 
module receives the current base plan from the communication module and 
returns a set of valid refinements over the base plan. 

^ Reasoning module: Agents coordination consists in evaluating the refinement 

f ’ ans and choosing the most promising one as the next base plan (see section 
. The reasoning module of each agent receives the refinement proposals of 
e agents and evaluates them according to the view of the MAP task of 
the respective agent. Hence, this module provides agents with facilities to 
perform the coordination process, allowing agents to reason about the different 
proposals and vote for the next base plan. 

In conclusion, the internal design of planning agents provides them with the 
basic capabilities required to solve MAP tasks. Agents use their internal com¬ 
ponents to interact with each other through the communication infrastructure, 
reason about plans and proceed with the next plan refinement. 

5. Planning model 

This section presents the MAP model upon which our planning architecture is 
based. It also describes the procedure followed by the agents for building and 
exchanging plans among them. 

The following subsections describe and formalize the main components of a 
MAP task and outline the Partial-Order Planning concepts used in the MAP 
algorithm (see section . In order to illustrate the formal definitions introduced 
in this section, we provide simple examples based on the transportation MAP 
task presented in section Also, for the sake of clarification of some definitions, 
we point out the reader to the figures of plans showed in section 

5.1. Formalization of a MAP task 

Definition 1. (MAP task) A MAP task is a tuple T = (A^,C1,V,A,I, ^,). 
AQ = {1,..., n} is a finite non-empty set of planning agents. O is a finite set of 
objects that model the elements of the planning domain over which the planning 
actions can act. V is a finite set of state variables that model the states of the 
world. Each state variable u S V is mapped to a finite domain of mutually 
exclusive values T^y. Each value in a state variables’s domain corresponds to an 
object of the planning domain, i.e. Vr G V, ^ O. When a value is assigned to 
a state variable, the pair variable-value acts as a ground atom in propositional 
planning. A is the set of deterministic actions of the agents. X is the set of values 
assigned to the state variables in V and represents the initial state of the MAP 
task T. Q is the set of goals of the MAP task that agents have to achieve; Q 
represents the values that the state variables are expected to take in the final 
state. 
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Information that agents have on the states of the world (problem states) is 
modeled through a set of ground atoms or fluents. This includes the initial state, 
X, and the goal state, Q. As opposite to STRIPS-like models (Pikes and Nilsson, 
1971), which apply negation by failure (only positive fluents are represented, the 
absence of a fluent implies its negation), we allow to explicitly represent both 
true and false information. Thus, our model adopts the open world assumption, 
considering that the information which is not explicitly stored in the internal 
model of agents is unknown to them. Again, this also refers to the information 
in the initial state, I, and the goals, Q. 

Definition 2. (Fluent) A ground atom or fiuent of the problem is a tuple of 
the form {v, d), where u G V and d GVy. A negative fluent is of the form {v, -^d). 
A positive fluent (u, d) indicates that the variable v takes the value d, while a 
negative fluent {v, -^d) indicates that the variable v does not take the value d. 

As stated in Definition a fluent relates a variable with one of the values 
in its domain. For instance, let (at tl) be a variable that refers to the position 
of a truck object tl in the example introduced in section Possible values for 
this variable are the cities cA, cB, cC, cD, cE and cF. Then, a positive fluent (at 
tl, cA) indicates that tl is in cA while a negative fluent (at tl, ^cA) indicates 
that tl is not in cA. 

In our model, agents are heterogeneous as they may have different knowledge 
and planning capabilities. In addition, they may have incomplete information on 
the MAP task as this can be distributed across agents. In this case, agents must 
cooperate with each other to solve the MAP task. Even though information is 
distributed across agents, there must be a subset of state variables shareable 
between agents in order to exchange the values of such variables and successfully 
communicate between each other. To denote the actions, goals, etc. of an agent 
i G AQ we will use the superscript notation a;* for any such aspect x. 

From the set of variables, V, of the MAP task, we distinguish V* as the set of 
variables managed by agent i, which includes the private variables, only known to 
agent i, and the public variables, shared with other agents. Thus, V = 

D* C Dy is the set of values of a variable u G V* that are visible to agent i. The 
information of the initial state of the MAP task, I, is modeled through a set 
of positive and negative fluents. This information is distributed among agents 
under the assumption that agents’ partial knowledge about I is consistent, i.e. 
there is not contradictory information among agents. Hence, X can be defined as 
X — possible to define MAP tasks in which all the agents have 

a complete view of the initial state X, i.e. Vz G AG, T* = X. 

For example, Agl and Ag2 are two transport agents in the MAP scenario of 
section]^ Initially, Agl knows that the truck tl is in city cA so the fluent (at 
tl, cA) is part of On the contrary, Ag2 does not know where tl is initially 

located, but it knows that the truck is not in city cB. Hence, the fluent (at tl, 
^cB) belongs to the initial state of Ag2. 

Each agent i G AG is associated with a set, of possible actions such that 
the set of actions of a planning task is defined as A = Uvie^e action a 

is said to be public if it is shared by two or more agents, i.e. a G A® A a G A-l, 
z yf j. a G A* is private to agent z iff a ^ yf z. An action a G A* denotes 

that agent z has the capability expressed in a. If a forms part of the final plan 
then agent i is also responsible of executing a. 
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Definition 3. (Planning rule or action) A planning rule or action a & A 

is a tuple {PRE{a), EFF{a)). PRE{a) = {pi,... ,Pn} is a set of fluents that 
represents the preconditions of a, while EFF{a) = {ei,...,em} is a set of 
operations of the form {v = d) or (v ^ d), v G V, d G Vy, that represent the 
consequences of executing a. 

An action a may belong to different agents, i.e. a G A'' and a G A^, i ^ j. 
Executing an action a in a world state S gives rise to a new world state S' 
generated as the result of applying EFF{a) over S. Particularly: 

— An operation {v = d) G EFF{a) implies the addition of a fluent {v,d) and a 
set of fluents {v,^d'), Vd' G Vy \ d' ^ d, to the world state S'. If {v,d') G S 
or (u, ->d) G S, d' ^ d, the operation {v = d) also implies that the fluents 
{v,d') or {v,^d) will not be present in S'. For example, suppose that agent 
Agl knows that the truck tl can be placed at the cities cA, cB, cC and cD, i.e., 

If knows a positive fluent (at tl, cA), it also 
knows the negative fluents (at tl, ^cB), (at tl, ^cC) and (at tl, ^cD). 

— An operation {v ^ d) G EFF{a) implies the addition of a fluent {v, -^d) to the 
world state S'. If {v, d) G S, the operation {v ^ d) also entails that the fluent 
(u, d) will not be present in S'. Note that the only existence of a fluent {v, ^d) 
in a state indicates that the value of the variable v is not known in such a 
state and, consequently, the rest of values in Vy, except for d, are unknown 
values. For example, if the fluent (at tl, ^cB) is in the world state of agent 
Ag2, then the agent only knows that the truck tl is not in city cB but the 
agent is not aware of the actual position of the truck. Thus, whether tl is in 
cA, cC or cD is unknown to Ag2. 

The set of preconditions of an action a, PRE{a), defines the fluents that 
must hold in a world state S for that a is applicable in this state. A positive 
precondition of the form (u, d) indicates that the fluent (u,d) must hold in S', 
while a negative precondition (u, -id) indicates that the fluent (u, -id) must hold 
in S. Note that the existence of a positive fluent (u, d) also implies the existence 
of a negative fluent (u, ^d') for the rest of values in the variable’s domain, i.e. 
(3(u, d) G S) ^ (Vd' G Vy, d' ^ d, 3(u, -d') G S). 

Additionally, agents use a utility function P to evaluate the quality of the 
plan proposals. For each agent i, P assigns a cost, cost{view''(A)) G Kg, to each 
plan proposal 11 according to the view that agent i has of that plan, view'{A). 
Finally, the private goals of an agent i, VQ', are fluents that agent i is interested 
in attaining. Private goals are encoded as soft constraints (Gerevini and Long, 
2006), as it is not mandatory that agents achieve them. 

5.2. Concepts on Partial-Order Planning 

Our MAP model can be regarded as a multi-agent refinement planning frame¬ 
work, a general method based on the refinement of the set of all possible plans 
(Kambhampati, 1997). An agent proposes a plan 11 that typically enforces some 
of the goals that have not yet been supported (see definition |^ ; then, the rest 
of agents collaborate on the refinement of 11 by solving some of these pending 
goals in 11. This way, agents cooperatively solve the MAP task by consecutively 
refining an initially empty plan. 

In this context, Partial-Order Planning (POP) (Barrett and Weld, 1994) 
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arises as a suitable approach to address refinement planning, since it is focused 
on solving the pending goals progressively. Consequently, agents in our MAP 
approach plan concurrent actions through the adoption of the POP paradigm. 
In the following, we provide some basic definitions concerning single-agent POP 
and its adaptation to a MAP context. 


5.2.1. Single-agent Partial-Order Planning 


Definition 4. (Partial plan) A partial plan is a tuple 11 = {A,OTZ,C£). 
A C A is the set of actions in 11. OTZ is a set of ordering constraints (^) on 

A. CL is a set of causal links over A. A causal link is of the form a /3 or 


{v,—id) ^ , A 1 n A A rt 1 

a —>■ p, where a G A and p G A are actions in A. a —>■ p indicates that there 
is an operation {v = d) such that v G V, d G T>v, {v = d) G EFF{a) and a fluent 


{v,d) G PRF{f3). a j3 indicates that there is a fluent {v,^d) such that 

V GV, d G Vy, {v,^d) G PRF{j3) supported by an operation (v ^ d) G FFF{a) 
or an operation {v = d') G FFF{a), d' GVy, d' ^ d. 


This definition of partial plan represents the mapping of a plan into a directed 
acyclic graph, where A represents the nodes of the graph (actions) and OTZ and 
CL are sets of directed edges representing the precedences and causal links among 
these actions, respectively. 

An empty partial plan is defined as Hq = (Aq, OTZq, CLq), where Aq contains 
do and af, the initial and final action of the plan, respectively. Oq and af are 
fictitious actions that do not belong to the action set of any particular agent. 
OTZq contains the constraint ag ^ af and CLq is an empty set. This way, a plan 
n for any given MAP task T will always contain the two fictitious actions such 
that PRF{ao) = 0 and FFF{ad) = T, PRF{af) = Q, and FFF{af) = 0; i.e. 
ao represents the initial situation of the MAP task T, and af represents the 
global goals of T. 

Assuming that ^ yf 0, an empty plan is said to be incomplete if the pre¬ 
conditions of Of/ are not yet supported through a causal link. The POP process 
is aimed at introducing causal links to support these preconditions, also called 
open goals. 


Definition 5. (Open goal) An open goal in a partial plan 11 = (A, OTZ, CL) 
is a fluent og of the form {v, d) or {v, ^d), such that v GV, d G Vy, og G PRF{d), 
d G A, and $a G A/a ^ d gCL, i.e., an open goal og is a precondition of ab 

action d in the plan 11 that is not yet supported by a causal link a ^ d ^ CL. 
openGoals{Il) denotes the set of open goals in 11. A plan is incomplete if it has 
open goals. Otherwise, we say it is a complete plan. 


As the POP search progresses, the causal links in a partial plan may become 
unsafe as a result of the introduction of a new action which is not ordered with 
respect to the causal link. These conflicts are called threats. 


Definition 6. (Threat) A threat in a partial plan 11 = (A, OTZ, CL) represents 
a conflict between an action of the plan and a causal link. An action 7 causes a 

threat over a causal link a d if ((^^ = d') G FFF{p) V (v yf d) G FFF{j)), 
where v G V, d G Vy, d' G Vy and d yf d', and there is neither an ordering 
constraint 7^0 nor d ^ 1- The action 7 will cause a threat over a causal link 



16 


A. Torreno et al 


of the form a P ii {v = d) G EFF{'j), where v G V, d G Vy, and there is 

neither an ordering 7^0 nor ^ ^ 7 . Threats{H) denotes the set of threats in 

n. 


A threat t G Threats(Il) can be solved by promoting or demoting the threat- 

{v^d) 

enmg action 7 with respect to the causal link a —)• p or a —>• p, i.e. mtroduc- 
ing an ordering constraint 7 ^ a or ^ ^ 7 . Threats and open goals are referred 
to as the flaws of a partial-order plan. The POP process is guided by solving the 
pending flaws of an initially empty partial plan. 

Figure |4[in section depicts a refinement plan for the example introduced 

in section ^ This refinement plan includes a causal link Init ^ —1 ^ load 
tl p3 cA. Suppose that a new action drive tl cA cB, that causes the truck 
tl to move from cA to cB, is added to the refinement plan and that this new 
action is not ordered with respect to load tl p3 cA. In this case, (at tl = 
cB) G FFF{Ar±'je tl cA cB). This effect causes a threat over the previous 
causal link, as it introduces a fluent (at tl, cB) that affects the value of the 
variable (at tl). This threat can be solved by introducing an ordering constraint 
load tl p3 cA ^ drive tl cA cB, i.e., demoting the threatening action drive 
tl cA cB with respect to the causal link. 

5.2.2. Multi-agent Partial-Order Planning 

Agents in our MAP model cooperate on the refinement of a base plan 11 by 
proposing refinement steps that solve some open goals in 11. This way, agents 
cooperatively solve the MAP task by consecutively refining 11, the initially empty 
base plan. 

Definition 7. (Refinement step) A refinement step 11* devised by an agent i 
over a base plan 11^, where g G openGoals{Ilg), is a triple 11* = (A*, Oii*, CL*), 
where A* C A is a set of actions and OR^ and CL* are sets of orderings and 
causal links over A*, respectively. 11* is a threat-free partial plan that solves g as 
well as all the new open goals of the form {v,d) or {v,^d) that arise from this 
resolution and can only be achieved by agent i, where {v G V*) A(v ^ V-l, Vj ^ i). 
That is, when solving a goal of a base plan, agents only accomplish the new 
open goals concerning their private fluents, leaving public goals unresolved. In 
other words, the refinement method only iterates over the public fluents. Let 
g G openGoals(ng) be a fluent of the form (v,d} or (v,^d}; an agent i proposes 
a refinement step over Ilg iff u G V*. 

In our MAP approach partial plans are multi-agent concurrent plans as two 
or more actions can be concurrently executed by different agents. Some MAP 
models adopt a simple form of concurrency: two concurrent actions are mutually 
consistent if none of them changes the value of a state variable that the other 
relies on or affects, too (Brenner and Nebel, 2009). We impose the additional 
concurrency constraint that the preconditions of two actions have to be consistent 
(Boutilier and Brafman, 2001) for these two actions to be mutually consistent. 
This definition of concurrency is straightforwardly extended to a joint action 

Oi ( 0 : 1 , ... , (Xjp ■ 

Definition 8 . (Mutually consistent actions) Two concurrent actions a G A* 
and P G Ai are mutually consistent if none of the following conditions holds: 
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— = d) G EFF{a) and 3{{v,d') G PRF{13) V {v,^d) G PRF{(3)), where 
u G V® n , d GVln Vi, d' G Vi and d ^ d', or vice versa. 

— 3(r! = d) G FFF{a) and 3((v = d') G FFF{P) y {v ^ d) £ FFF{j3)), where 
u G V® n , d G h* n vi, d' G vi and d ^ d', or vice versa. 

— 3(z;, d) G PRF{a) and 3{{v,d') G PRF{j3) V {v,^d) G PRF{P)), where 
r; G V® n V-^, d G h® n vi, d' G and d ^ d', or vice versa. 

Going back to our example in section two concurrent actions drive tl 
cA cB, planned by agent Agl, and drive tl cA cC, planned by agent Ag2, 
are mutually inconsistent as (at tl = cB) G FFF{drive tl cA cB) and (at 
tl, cC) G PRE{dr±ve tl cC cB) (the first condition in Definition holds). 
Concurrent actions drive tl cA cB and drive tl cA cC are also mutually in¬ 
consistent as (at tl = cB) G EFF{dr±ve tl cA cB) and (at tl = cC) G 
EFF{drive tl cA cC) (second condition holds). Finally, concurrent actions 
drive tl cA cB and drive tl cC cB are mutually inconsistent as (at tl, 
cA) G Pi?£'(drive tl cA cB) and (at tl, cC) G Pi?£'(drive tl cC cB) (third 
condition holds). 

As agents address concurrency inconsistencies through the detection of threats 
over the causal links of their plans, concurrency is ensured among private actions 
since a refinement step put forward by an agent is always a threat-free plan. 
However, concurrency issues between two public actions introduced by different 
agents do not arise until their preconditions are fully supported through causal 
links. This way, it is not possible to ensure that two concurrent actions are mu¬ 
tually consistent until their preconditions are fully supported. Thus, our notion 
of multi-agent concurrent plan distinguishes between public and private actions 
when dealing with concurrency. 

Definition 9. (Multi-agent concurrent plan) A partial plan H = (A, OTZ, 
CC) is a multi-agent concurrent plan if for every pair of unequal, concurrent, 
public actions a and /3, a ^ j3, Vpa G PRE{a),Pa ^ openGoals{)l), Vp/? G 
PRE{P),pi 3 ^ openGoals(Il), a and /3 are mutually consistent. 

Definition 10. (Refinement plan) A refinement plan H devised by an agent 
i over a base plan Hg is a concurrent multi-agent plan that results from the 
composition of Hg and a refinement step H* proposed by agent i. H is defined as 
n = Hg o n*, where o represents the composition operation. 

Thus, an agent i can build a refinement plan H upon a base plan Hg by 
composing Hg and a refinement step H* that solves at least g G openGoalsilig), 
i.e. n = HgoH*. As previously mentioned, refinement steps are always threat-free 
and their actions are mutually consistent. Hence, if a refinement step brings about 
a concurrency inconsistency or a threat on the composite plan, the proposer 
agent is responsible for addressing such a flaw. If an agent is not able to come up 
with a consistent refinement plan, then it refrains from suggesting it. In case no 
refinements for a base plan can be found, the base plan is said to be a dead-end 
plan. 

Definition 11. (Dead-end plan) A plan H is called a dead-end plan if35G 
openGoals{Jl) and there is no refinement step H* such that g ^ openGoals{Jl o 
H*); that is, no refinement step solves the open goal g. 
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Definition 12. (Solution plan) A multi-agent concurrent plan 11 is a solu¬ 
tion plan for a planning task T if openGoals{ll) = 0 (11 is a complete plan), 
Threats{U) = 0, and every pair of actions a,/3 € 11 are mutually consistent. 

That is, a solution plan is a complete multi-agent concurrent plan. Note 
that we require 11 to be a complete plan so it cannot have pending open goals. 
Consequently, the preconditions of the fictitious final action a f will also hold thus 
guaranteeing that 11 solves the MAP task T. For instance. Figure in section 
1^ shows a solution plan for the MAP task presented in section The different 
shapes of the actions indicate which agent has proposed them. The solution plan 
in Figure is a complete, concurrent plan to which all the agents in the MAP 
task have contributed. 


6. Planning language for MAP tasks 

In our MAP system, we define the agents’ planning tasks through several speci¬ 
fication files. These files encode the information of the agent on the MAP task, 
namely the variables, V®; the objects associated to the variables, O*; the planning 
actions, A*; and the initial state of the agent, Tb All this information is written 
in a planning definition language. 

Traditionally, planning has been regarded as a single-agent problem, where 
only one centralized planning entity is required. MAP presents new requirements 
and challenges that are not present in classical, centralized planning. Planning 
agents in our MAP model can withhold their private information, and decide 
which information to share with the rest of agents. In addition, planning agents 
can have private individual objectives besides the goals of the planning task. 
Therefore, the planning language must provide support to allow us to define 
shareable information and private goals. 

Planning definition languages have experienced a remarkable evolution over 
the last years, continuously increasing their expressivity through the addition of 
new features. Our MAP language is based on PDDL3.1 (Kovacs, 2011), the most 
recent version of PDDL (Ghallab, Howe, Knoblock, McDermott, Ram, Veloso, 
Weld and Wilkins, 1998), which was introduced in the context of the 2008 Inter¬ 
national Planning Competition. Unlike its predecessors, that model a planning 
domain through logical predicates, PDDL3.1 also incorporates state variables 
by adding fluents that map a tuple of objects to an object of the planning task. 
We have extended the PDDL3.1 language with some new structures to model 
the multi-agent features of a planning task. 

In single-agent PDDL language, the user writes two files, one containing the 
domain of the task and another one containing the data of the problem to be 
solved. The domain file describes the planning actions, the types of objects and 
the state variables of the task, while the problem file details the current objects 
of the task, the initial state (the initial values of the state variables) and the 
task goals. These files have a similar structure to their PDDL counterparts, and 
reflects the additional information required by MAP tasks. 

In our MAP system, each agent has a domain and a problem file that model, 
respectively, the typology of the planning agent and its particular vision of the 
MAP task. The domain and problem files also include the information that is 
shared among agents. The shared-data structure allows the problem designer 
to define which fluents will be shared by each agent and with whom. Through 
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this structure, the designer can define the incomplete information of the agent. 
This way, the domain knowledge of the agents can be modeled (or specified) 
from a complete unawareness to a full visibility of the domain. Additionally, 
since agents in MAP may have both global and local goals, this information is 
modeled through the structures global-goal and private-goal. Finally, we 
have included an additional multi-functions structure in order to simplify the 
specification of fluents in the initial state of an agent. 

The following subsections analyze the structures that cover the requirements 
of MAP domains, i.e. modeling the data shared among agents, and the definition 
of local and global goals. The last subsection provides an example that describes 
the encoding of the MAP task introduced in section with our language. 


6.1. Shared data 


The shared-data structure, located on the agent’s problem file, determines 
which fluents are shareable and with which agent or agents they will be shared. 
As shown in section this structure directly affects the initial information ex¬ 
change that agents perform before planning, and it also defines the partial view 
of the planning task of each agent. 

As agents only exchange fluents, in the : shared-data structure the problem 
designer specifies the fluents that the agent can share and with which agents. 
The shared-data structure has the following BNF syntax: 


<shared-data-def> 
<share-def> 
<agent-def> 

<agent> 

< a t om-fo rmu 1 a-de f > 
<atom-formula-def> 
<predicate> 

<object-fluent> 

<object> 

<element> 
<variable> 
<constant> 
<typed-list(x)> 


(:shared-data <share-def>+) 
(<atom-formula-def>+ [- <agent-def>?]) 
<agent> | (either <agent> <agent>+) 
<name> 

(<predicate> <typed-list(element)>) 

(= <object-fluent> <object>) 

<name> 

(<name> <object>*) 

<name> 

<variable> | <constant> 

?<name> 

<name> 

X* 


As the BNF syntax shows, it is possible to define fluents or predicates within 
the : shared-data section and associate them to one, some or all the agents in 
the system (if agent is not specified, the predicates or fluents are shared with 
all the agents). 


6.2. Private and global goals 

A particularity of the MAP approach when compared to traditional planning is 
the fact that agents have private and global goals. To reflect this information in 
the model, the private-goal and global-goal structures have been included 
into the problem files. Similarly to the goal section in PDDL3.1, goals can be 
modeled through predicates or fluents. The private-goal and global-goal 
structures use the following BNF syntax: 
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<private-goal-def> 
<global-goal-def> 
<predicate-def> 
<predicate-def> 
<predicate-def> 

< a t om-fo rm-de f > 
<atom-form-def> 
<predicate> 

<object-fluent-def> 
<object> 

<element> 

<variable> 
<constant> 
<typed-list(x)> 


(:private-goal <predicate-def>) 

(:global-goal <predicate-def>) 

< at om-fo rmu1a-de f > 

(and <atom-form-de f> <atom-form-de f>+) 
(or <atom-form-def> <atom-form-def>+) 
(<predicate> <typed-list(element)>) 

(= <object-fluent-def> <object>) 

<name> 

(<name> <object>*) 

<name> 

<variable> | <constant> 

?<name> 

<name> 

X* 


As shown in the BNF syntax description, both global and local goals are 
described as conjunctions or disjunctions of fluents and predicates, or rather as 
a single fluent or predicate. 


6.3. Encoding example 

This subsection describes the encoding of the MAP task presented in section 
with our MAP language. This MAP task describes a transportation and storage 
scenario in which two agents (Agl and Ag2) take the role of transport agencies 
and an agent (Ag3) manages a storage facility. Transport agents deliver packages 
through a network of cities, while the warehouse agent stores and loads the 
packages in trucks. In the following, we provide a description of the domain and 
problem files of the agents for this task, stressing the specification of shareable 
information. 

Planning agents receive two different description flies, namely the domain 
and problem file. The domain file contains a general description of the capa¬ 
bilities of the agent, including the actions that the agent can perform and the 
predicates and functions it can manage. All agents of the same type share the 
same domain file, e.g. transport agents Agl and Ag2 in this example receive the 
same domain file. The problem file models the concrete problem assigned to each 
agent, including a description of the objects of the task, the initial situation and 
the global goals of the task as well as private goals of the agent. Each agent 
receives its particular problem file. 

6.3.1. Domain files 

The domain file for transport agents specifies bidirectional links among cities, 
which allow trucks to move from one city to another. Trucks can only travel 
within the cities included in their working areas, depicted in Figure with two 
circles. This way, transport agents have to interact and cooperate in order to de¬ 
liver packages to a different area. The domain file for transport agents is modeled 
as follows: 
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(define (domain Transport) 

(: requirements ityping :equality ;fluents) 

(:types truck package agent city - object 

raw-material final-product - package) 

(:predicates (empty ?c - city)) 

(: functions (at ?t - truck) - city 

(pos ?p - package) - (either city truck) 

) 

(: multi-functions (link ?c - city) - city 
(area) - city 

) 

(: action load 

:parameters (?t - truck ?p - package ?c - city) 

:precondition (and (member (area) ?c) (= (at ?t) ?c) (= (pos ?p) ?c)) 

:effect (and (assign (pos ?p) ?t)(empty ?c)) 

) 

(: action unload 

:parameters (?t - truck ?p - package ?c - city) 

:precondition (and (empty ?c)(member (area) ?c) 

(= (at ?t) ?c) (= (pos ?p) ?t)) 

:effect (and (assign (pos ?p) ?c)(not (empty ?c))) 

) 

(: action drive 

:parameters (?t - truck ?cl ?c2 - city) 

:precondition (and (member (area) ?cl)(member (area) ?c2) 

(member (link ?cl) ?c2) (= (at ?t) ?cl) ) 

:effect (assign (at ?t) ?c2) 

) 

) 


The domain file shown above is structured similarly as a regular PDDL3.1 
file. The main sections of the file are highlighted in bold. The ; requirements 
section indicates the PDDL features that have been used to encode the domain 
information. ; types describes the object-type hierarchy of this particular do¬ 
main. As it can be seen, the planning domain of transport agents includes four 
different types of objects, namely truck, agent, city and package. A package 
can be either a raw-material or a final-product. 

Structures ; predicates, ; functions and ;multi-functions define the state 
variables used in the transport domain. During the planning process, these vari¬ 
ables will be instantiated to objects defined in the transport agents’ problems, 
thus giving rise to the fluents that will be used throughput the planning process. 
For instance, let us consider the function (at ?t - truck) - city, where (at 
?t) is a state variable and city is the type of its domain values. Given a truck 
object tl and a city object cl, the previous function will result in a fluent of 
the form (= (at tl) cl), which indicates that tl is located at cl. 

The domain file of transport agents include the following predicates, functions 
and multi-functions: empty is a predicate to indicate whether a city is empty or 
already contains a package (a city can only have one package simultaneously); 
function at returns the city in which a certain truck is placed; function pos 
describes the position of a package, either a truck or a city; multi-function 
link returns the outcoming connections (roads) from a certain city; and curea 
describes the working area of an agent in terms of the cities it can drive a truck 
to. 

The last portion of a PDDL3.1 domain file defines the abilities of the agent, 
i.e., the actions it can perform. Actions are described through its parauneters 
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(objects that take part in the action), preconditions (predicates and func¬ 
tions that must hold for the action to be applicable) and effects (predicates 
and functions that describe the consequences of applying the action). As in the 
case of predicates, functions and multi-functions, actions are described through 
state variables. In particular, preconditions encode queries on fluents that check 
whether a variable takes on a particular value, and effects encode assignment 
operations on fluents to make a state variable take on a value. 

As described in the domain file, transport agents can perform three different 
actions: load and unload a package into/from a truck, and drive a truck from 
a city to another one of the agent’s area. 

The domain file for warehouse agents is similar to the classical blocksworld 
domain, in which packages can be stacked and unstacked on/from the table or 
other packages. In this case, only one pile of packages can be stacked on the 
table, and there are two types of packages, raw materials and final products. 
The transportation and storage scenario depicted in Figure includes two final 
products (packages pi and p2) and a raw material (package p3). The warehouse 
agent delivers final products to the city in which the warehouse is placed (the 
exchange city, cF in Figure[^, and acquires raw materials that are unloaded from 
the trucks in the exchange city. Following, we show a sketch of the warehouse 
domain file encoding: 

(define (domain Warehouse) 

(: requirements :typing :equality :fluents) 

(:types package agent city table hoist - object 
raw-material final-product - package) 

(:predicates (empty ?c - city) 

(clear ?p - (either package table hoist)) 

(exchange-city ?c - city) 

) 

(:functions (pos ?p - package) - (either city package table hoist)) 

(: action acquire 

:parameters (?p - raw-material ?c - city ?h - hoist) 

:precondition (and (= (pos ?p) ?c)(clear ?h)(exchange-city ?c)) 

:effect (and (assign (pos ?p) ?h) (not (clear ?h) ) (empty ?c)) 

) 


) 

As the transport agents, warehouse agents manage city, hoist and package 
objects. Additionally, warehouse agents consider table and hoist objects. The 
hoist is used to deliver and acquire packages, while the table is used to 
stack and unstack packages within the warehouse. 

Warehouse agents perform the four actions indicated above: they can stack 
and unstack a package on top/from a clear table or package; and can also 
acquire and deliver a package from/to the exchange-city by using a hoist. 
The sketch of the domain file illustrates the encoding of the acquire action. 

6.3.2. Problem files 

Each agent receives its own problem file that models the particular objects man¬ 
aged by the agent, the initial situation known to the agent and the global and 
private goals that the agent must achieve. Moreover, the problem files include 
the definition of the shareable fluents and with which agents they can be shared. 
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We now explain the problem file of transport agent Agl (this problem will be 
later used to illustrate the construction of the dis-RPG). Problem files describe 
the initial state of the task by including both the positive and negative infor¬ 
mation known by the agent. This way, the information not represented in the 
problem file is unknown to the agent. Agl’s problem file is encoded as follows: 

(define (problem Agl) 

(: domain Transport) 

(: objects 

Agl Ag2 Ag3 - agent 
tl - truck 

cA cB cC cD cE cF - city 
p3 - raw-material 
pi p2 - final-product) 

(: shared-data 

(empty ?c - city) - (either Ag2 Ag3) 

((at ?t - truck) - city) 

((pos ?p - package) - (either city truck)) - Ag2 
((pos ?p - package) - city) - Ag3 

) 

(: init 

(empty cB) (empty cC) (empty cD) (not (empty cA) ) 

(= (at tl) cA) (not (= (at tl) cB) ) (not (= (at tl) cC) ) 

(not (= (at tl) cD) ) (= (pos p3) cA) (not (= (pos p3) cB) ) 

(not (= (pos p3) cC) ) (not (= (pos p3) cD) ) 

(= (link cA) {cB cC}) (not (= (link cA) {cA cD}) ) 

(= (link cB) {cA cC}) (not (= (link cB) {cB cD}) ) 

(= (link cC) {cA cB cD}) (not (= (link cC) {cC})) 

(= (link cD) {cC}) (not (= (link cD) {cA cB cD}) ) 

(= (area) {cA cB cC cD}) (not (= (area) {cE cF}) ) 

) 

( :global-goal (and (= (pos pi) cA)(= (pos p3) cE))) 

) 


Sections of the problem file are also highlighted in bold. A domain file starts 
with a description of the : objects that the agent manages. As shown in the 
code, agents are represented as objects. Agl knows that there is a truck tl in 
the task, and it has knowledge of six different cities, although it only manages 
the four cities included in its working area (see Figure Q. The agent also knows 
that there are three packages in the MAP task, the final-products pi and p2 
and the raw-material p3. 

The : shared-data section is a key aspect of our MAP language, as it defines 
the information shareable by the agents and directly affects their knowledge of 
the task. The predicates and functions defined in this structure are the patterns 
of the fluents that the agent regards as shareable with other agents. For instance, 
Agl’s ; shared-data section includes the following pattern: (empty ?c - city) 
- (either Ag2 Ag3) . This pattern indicates that Agl will share the fluents that 
match the pattern (empty ?c - city) with both Ag2 and Ag3. Given that Agl 
knows the cities cA, cB, cC, cD, cE and cF, fluents as (= (empty cA) true) or (= 
(empty cD) false) match the pattern, and Agl shares this information with 
Ag2 and Ag3. 

The : init section describes the initial state of Agl, i.e., the initial situation of 
the world known to Agl. It is defined with predicates like (empty cB), functions 
like (= (at tl) cA)) and multi-functions like (= (link cA) {cB cC}), that 
hold in the initial situation. The initial state includes both positive and negative 
information. For instance, the function (not (= (at tl) cC)) indicates that 
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Agl knows that truck tl is not initially placed at city cC. The information 
not included in the initial state is considered unknown to Agl. 

While the initial state contains predicates, functions and multi-functions, 
internally the system treats all of them as fluents. For instance, a predicate 
(empty cB) is internally converted into a fluent (= (empty cB) true), while 
functions like (= (at tl) cA) are already in the form of fluents. Multi-functions 
are used to easily define multiple functions through a simplified notation. The 
conversion into fluents is straightforward: given a multi-function (= (link cA) 
cB cC) , we generate the fluents (= (link cA cB) true) and (= (link cA cC) 
true). 

Finally, the : global-goal structure shows the global objective of the MAP 
task. In this case, the goal is to transport the raw-material p3 to city cF, and 
to deliver a final-product to city cA. Notice that, in this example, there is 
not a : private-goal section. 


7. MAP algorithm overview 

This section summarizes the main stages of the MAP algorithm followed by the 
agents to devise, exchange and select refinement plans to come up with a solu¬ 
tion for the MAP task. Agents follow a procedure that integrates planning and 
coordination, allowing agents to solve both strongly-related and loosely-coupled 
problems without losing generality. Agents perform an individual Partial-Order 
Planning (POP) search to build refinements over the current base plan, while 
one of the agents leads the process of gathering the new refinement plans and 
selecting the next base plan. 


Algorithm 1 Overview of the MAP algorithm 
Initial information exchange 

repeat 

Individual refinement process 
Coordination process 

until a solution plan is found or the search space is completely explored 


Algorithm [2 shows the main steps of the MAP algorithm. The stages of the 
algorithm are outlined as follows: 

— Initial information exchange: The algorithm starts with an initial com¬ 
munication stage by which agents exchange the shareable information on the 
planning domain in order to generate the data structures that will be used 
in the subsequent planning process. Agents take advantage of the exchanged 
information to build a distributed Relaxed Planning Graph, which provides 
them with their partial view on the MAP task. 

— Resolution process: Once agents have exchanged the shareable information 
and the distributed Relaxed Planning Graph is computed, they start the iter¬ 
ative resolution process by which they explore the search space until they find 
a solution for the MAP task. As shown in Algorithm this process comprises 
two different interleaved stages, an individual planning process by which agents 
devise refinements over a centralized base plan and a coordination process to 
exchange the new refinement plans and to select the next base plan: 
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Individual refinement process: Agents individually refine the current 
base plan of the MAP system. Each planning agent is provided with an 
internal POP system. The classical POP algorithm has been adapted to a 
MAP context in order to o btain valid refinement plans over an incomplete 
base plan (see section 9.1). 

Coordination process: Agents communicate and exchange the new re¬ 
finement plans over the current base plan. Later, they jointly evaluate these 
refinement plans and select the most promising one as the next base plan. 


The following sections detail the two main stages of the MAP algorithm. 
Section describes the initial information exchanging stage performed by the 
agents, while section details the resolution process, including both the coordi¬ 
nation process and the individual construction of the refinement plans. 


8. Initial information exchange 


Prior to the resolution process itself, agents perform a preliminary stage to share 
public planning information effectively. This initial stage builds a distributed 
Relaxed Planning Graph (dis-RPG), whose construction is inspired by the ap¬ 
proach in (Zhang et al., 2007). Unlike the proposal in (Zhang et ah, 2007), which 
stops the graph construction once all the problem goals appear in the graph, our 
procedure builds a complete dis-RPG by maintaining the incomplete information 
of the agents, so they only exchange the information defined as shareable in the 
input files (see section l6|. This section describes in detail the dis-RPG building 
process and subsection |8.1| provides a trace based on the MAP task presented in 
section that illustrates this process. 

The dis-RPG provides the agents with valuable planning information that 
will be used throughout the refinement planning process: 


Agents exchange the fluents defined as shareable in t he : shared-data section 
of the MAP domain definition files (see subsection 6.1). Fluents are labeled 
with the list of agents that can achieve them, giving each agent a view of the 
possible interactions that can arise at planning time with other agents. 

An estimate of the best cost to achieve each fluent is computed. This informa¬ 
tion is used to design heuristics to guide the refinement planning process. 


Following Algorithm the first step of the dis-RPG construction consists 
in computing the initial RPG for each agent i, RPG^, taking only into account 
the fluents and actions initially known to the agent. Agents compute this initial 
planning graph by following the procedure presented in (Hoffmann and Nebel, 
2001). The RPG consists of a set of alternating fluent and action levels. The 
first fluent level contains the fluents that are part of the initial state, and the 
first action level includes all the actions whose preconditions appear in the first 
fluent level. Fluents that are part of the effects of these actions (and have not 
been included in the first fluent level) are placed in the second fluent level, and 
actions whose preconditions are included in the two prior fluent levels of the 
graph (and are not in the first action level) are stored in the second action level. 
By following this procedure, the RPG is expanded until no new fluents are found. 
This way, the level of the graph in which an action or fluent appears gives an 
estimate of the cost of achieving such an action or fluent. 

Once all agents have computed their initial RPGs, the iterative composition 
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Algorithm 2 Dis-RPG construction for an agent i 

Build initial RPG^ 

repeat 

\/j i, i sends j shareable fluents G RPG^ of the form {v, d) or 

{v, -^d), where u G V* n and d G 'DlD'Dl 

Vj i, i receives from j shareable fluents SF^^'‘ G RPG^ of the form {v, d) 
or {v, -^d), where u G V® n and d G'DlH'Dl 
RF^ G- 0 

Vj ^ i, RF* G- RF^ U SF^^^ 
for all received fluents / G RF^ do 
if / ^ i?PG* therr 
Insert f in RPG^ 
levelRPGiif) G- level{f) 
end if 

if (/ G RPG’’) A {levelppGi{f) > level{f)) then 
levelppGiif) G- levelQ) 

end if 
end for 
Expand RPG^ 
until PF* = 0 


of the dis-RPG begins. As depicted in Algorithm]^ after computing the initial 
RPG^, agent i executes the first iteration of the algorithm and exchanges the 
fluents and actions of its RPG^ with the rest of agents. Agents only exchange the 
fluents def ined as shareable in the : sheired-data structure of the input flies (see 
subsection |6.l| . Agent i sends agent j the set of fluents SF^^^ that are visible to 
agent j, i.e., the fluents in RPG' of the form (u, d) or {v, ^d), where u G V* n 
and d G PI nVi. Likewise, agent i will receive from the rest of agents j ^ i the 
shareable fluents of their RPG^ that are visible to agent i. 

Agent i updates its RPG^ accordingly with the new fluents received from the 
rest of agents. We will refer to these fluents as PF* (see Algorithm^. If a fluent 
/ G PF* is not in PPG* then it is stored according to level{f). If / is already in 
RPG^, its level in the graph is updated if levelppQiif) > level{f). Hence, agents 
only store the best estimated level to reach each fluent, placing each fluent at 
the lowest possible level of the graph. After updating PPG*, agent i expands it 
by checking wether the new inserted fluents trigger new actions in RPG^ or not. 
The fluents produced as effects of these new actions will be shared in the next 
information exchange iteration. The RPG expansion procedure also updates the 
existing actions by placing them at a lower action level if their preconditions 
have been updated. 

Since agents only exchange those fluents defined as shareable, the dis-RPG 
process gives each agent a different view of the planning task, so no agent handles 
a complete representation of the dis-RPG. In contrast, each agent i maintains 
its own internal RPG^, whose information depends on the fluents other agents 
have shared with it, which makes each agent have its own, partial view of the 
planning task. Thus, agents design plans under incomplete information, as they 
are partly aware of the information on the planning task. 

The dis-RPG process finishes when agents do not receive more fluents from 
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FO_ 

[2] (empty cB) T 
[2] (empty cD) T 
[2] (empty cE) T 
[2] (empty cF) T 
[2] (link cB cE) T 
[2] (link cD cE) T 
[2] (link cD cF) T 


[2] (link cE cB) T 
[2] (link cE cD) T 
[2] (link cF cD) T 
[2] (area cB) T 
[2] (area cD) T 
[2] (area cE) T 
[2] (area cF) T 


Table 1. Initial RPG built by agent Ag2 


FO 


AO 

FI 

A1 

[l](at tl) cA 
[1] (empty cA) F 
[1] (empty cB) T 
[1] (empty cC) T 
[1] (empty cD) T 
[l](link cA cB) T 
[l](link cA cC) T 
[l](link cB cA) T 
[l](lmk cB cC) T 

[l](pos p3) cA 
[l](link cC cA) T 
[l](lmk cC cB) T 
[l](lmk cC cD) T 
[l](lmk cD cC) T 
[1] (area cA) T 
[1] (area cB) T 
[1] (area cC) T 
[1] (area cD) T 

load tl p3 cA 
drive tl cA cB 
drive tl cA cC 

[l](pos p3) tl 
[ij (empty cA) T 
[l](at tl) cB 
[1] (at tl) cC 

unload tl p3 cB 
unload tl p3 cC 
unload tl p3 cA 
drive tl cB cA 
drive tl cB cC 
drive tl cC cA 
drive tl cC cB 
drive tl cC cD 


F2 

A2 

F3 

A3 

[1] (empty cB) F 
[ll (empty cC) F 
[l](at tl) cD 
[l](pos p3) cB 
[l](pos p3) cC 

load tl p3 cB 
load tl p3 cC 
unload tl p3 cD 
drive tl cD cC 

[1] (empty cD) F 
[l](pos p3) cD 

load tl p3 cD 


Table 2. Initial RPG built by agent Agl 


the others. Following, agents start the resolution process to jointly devise a so¬ 
lution plan. 


8.1. dis-RPG example 

In order to illustrate the dis-RPG stage of the MAP algorithm, this section 
provides an example trace based on the transportation and storage MAP task 
introduced in section]^ The planning agents receive the input files presented in 
subsection |6.3| and start the MAP algorithm by building the dis-RPG. 

In the first stage of Algorithm each agent individually generates an initial 
RPG, according to its problem file. To illustrate this stage of the process, we 
focus on the initial RPGs built by the transport agents Agl and Ag2. 

Table shows the initial RPG calculated by agent Ag2. The numbers in 
brackets indicate the agents that can generate the fluent, while the values T and 
F stand for true and false, respectively. Ag2 does not know the position of the 
packages and the truck because they are initially located out of its working area 
(see Figurein section]^. Therefore, its initial RPG only includes FO, the first 
level of fluents, which stores the fluents on the initial state of Ag2. The initial 
RPG of Ag2 does not contain any action level because there are no applicable 
actions, that is, their preconditions do not hold in FO. 

Agent Agl does know the position of the package p3 and the truck tl, and 
consequently, it can compute a much larger initial RPG (see Table [^. Notice that 
the level AO includes the actions whose preconditions are satisfiM in FO, while 
FI stores the fluents that are part of the effects of the actions in AO and are not 
in FO. For instance, the action drive tl cA cB, at level AO, has the following 
preconditions: (= (area) cA), (= (area) cB), (= (link cA cB) true) and 
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FO 


AO 

FI 

Al 

[1, 2](empty cB) T 
[1, 2](empty cD) T 
[2] (empty cE) T 
[2] (empty cF) T 
[2] (link cB cE) T 
[2] (link cD cE) T 
[2] (link cD cF) T 
[1] (empty cA) F 
[l](pos p3) cA 

[2] (link cE cB) T 
[2] (link cE cD) T 
[2] (link cF cD) T 
[2] (area cB) T 
[2] (area cD) T 
[2] (area cE) T 
[2] (area cF) T 
[1] (at tl) cA 
[2, 3] (empty cF) T 

load tl p3 cA 
drive tl cA cB 
drive tl cA cC 

[1] (empty cA) T 
[1, 2](at tl) cB 
[l](at tl) cC 
[1, 2](pos p3) tl 

unload tl p3 cB 
unload tl p3 cC 
unload tl p3 cA 
drive tl cB cA 
drive tl cB cC 
drive tl cC cA 
drive tl cC cB 
drive tl cC cD 


F2 

A2 

F3 A3 

[1, 2](empty cB) F 

load tl p3 cB 

[l,2](empty cD) F load tl p3 cD 

[1] (empty cC) F 

load tl p3 cC 

[2] (at tl) cF 

[1, 2](at tl) cD 

unload tl p3 cD 

[1, 2](pos p3) cD 

[2] (at tl) cE 

drive tl cD c(J 

[2] (pos p3) cE 

[1, 2](pos p3) cB 


[2] (empty cE) F 

[l](pos p3) cC 
[2, 3](pos pi) cF 
[1, 2](empty cF) F 


[2, 3] (pos p2) cF 


F4 

[2](pos p3) cF 

[1, 2](pos pi) tl 
[1, 2](pos p2) tl 


A4 _ 

unload tl pi cB 
unload tl pi cC 
unload tl pi cD 
unload tl pi cA 
unload tl p2 cB 
unload tl p2 cC 
unload tl p2 cD 
unload tl p2 cA 


F5 _ 

[l](pos pi) cA 

[1, 2](pos pi) cB 

[1] (pos pi) cC 
[1, 2](pos pi) cD 

[2] (pos pi) cE 
[l](pos p2) cA 
[1, 2](pos p2) cB 

[1] (pos p2) cC 
[1, 2](pos p2) cD 

[2] (pos p2) cE 


Table 3. Final dis-RPG as viewed by agent Ag2 


A5 _ 

load tl pi cB 
load tl pi cD 
load tl p2 cB 
load tl p2 cD 
load tl pi cC 
load tl pi cA 
load tl p2 cC 
load tl p2 cA 


(= (at tl) cA). As Tableshows, these fluents are at FO, which triggers the 
action drive tl cA cB at AO. 

Once agents have built their initial RPGs, they start the iterative dis-RPG 
building process by exchanging the shareable fluents in their RPGs. 

In subsection |6.3.2[ we show the ; shcured-data section of Agl, which shares 
with Ag2 fluents that match the following patterns: (empty ?c - city), ((at 
?t - truck) - city) and ((at ?t - truck) - city). The fluents shared by 
Agl and Ag2 are marked in red in Table Ag2 also sends its shareable fluents 
to the rest of agents and stores the fluents received from other agents. 

Agents expand their RPGs by checking if the fluents they have received trig¬ 
ger new actions in the graph. The process carries on until no new fluents appear 
in the dis-RPG. As each agent has a different : shared-data section, the infor¬ 
mation will vary from one RPG to another, giving each agent a different and 
incomplete view of the dis-RPG and the MAP task itself. 

Table shows the final dis-RPG of the transportation scenario as seen by 
agent Ag2. As it can be observed, the dis-RPG provides both an estimate of 
the cost of achieving each fluent (this cost corresponds to the level at which the 
fluent appears), and the set of agents that achieve that fluent in the RPG. 


9. Resolution process 

After the information exchange, agents initiate the resolution process (see Algo¬ 
rithm^, which comprises two interleaved stages: the individual refinement stage 
and the coordination stage. The first stage involves agents building individual 
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refinements over a centralized base plan by using a POP. In the second stage, 
agents follow a coordination process to gradually build a joint solution plan for 
the MAP task, exchanging and evaluating the refinements generated individually 
and selecting the most promising one in order to reach a solution. 


Algorithm 3 Resolution process for an agent i 
n f— Ho 
i? = 0 

repeat 

Select open goal g G openGoalsiJl) 

Refinements^ijlg) f— Refine base plan Ilg individually 
Vj ^ t, send Refinements''{Iig) to agent j 
yj ^ i, receive Refinements^ijlg) 

Refinements{Ilg) f— Refinements’‘(Ilg) 

Vj yf i, Refinementsijlg) f— Refineraents{Jlg) U Refinements^^Hg) 
Evaluate Refinements{Ilg) 
i? f— i? U Refinements{Ilg) 

Vote for the best plan 11* G i? 

n ^ n* 

if openGoals(n) = 0 then 
return 11 
end if 
until i? = 0 


9.1. Individual refinement stage 


A planning agent i executes its individual POP process in order to refine the 
current base plan 11. As shown in Algorithm agent i refines 11 by solving a 
particular open goal g G openGoals{Jl), thus obtaining a set of valid refinement 
plans over Ilg, Refinements''{Iig). 


Our definition of refinement plan (see subsection 5.2.21 states that a refine¬ 
ment plan n* of an agent i over a base plan 11 solves one of its open goals 
g G openGoalsiJl), as well as all the private open goals g* of the form {v,d) or 
{v,^d) that arise from this resolution, where u G V* Ad G VI A m J i,v t 
V-^) V (Vj J i,d ^ Vi)) A (5* ^ openGoalsJl)). 

We have designed a customized version of the classical POP algorithm com¬ 
pliant with the requirements introduced by the MAP approach. Our POP system 
is able to start the search process from any given base plan, rather than starting 
with an empty plan as in a traditional POP process. In addition, the POP is 
aimed at building refinement plans, rather than complete solution plans. 


9.2. Coordination process 

The coordination process is based on a democratic leadership in which a lead¬ 
ership baton is scheduled among the agents (initially, the baton is randomly 
assigned to one of the participating agents). The resolution process interleaves 
the coordination process with the individual refinement stage. A coordination 
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iteration is led by the agent which has the baton (baton agent). Once the coor¬ 
dination iteration is completed, the baton is handed over to the following agent. 

Algorithm depicts the main steps of the coordination process. After the 
individual refinement stage, agents exchange the refinement plans they have 
elicited over the current base plan 11. Following, agents receive the refinement 
plans of the other agents and evaluate them according to their view of the plan¬ 
ning task. Agents apply a voting process to adopt the most promising plan as 
the next base plan 11, and check if the selected plan is a solution. Otherwise, 
agents choose a new open goal of the plan g G openGoals{Il) and each agent 
i starts a new individual refinement stage to compute the refinements over 11, 
Refinements'^ (Ilg). 

In the first step of the coordination process, the individual refinement plans 
are exchanged between agents for their evaluation. An agent i sends the re¬ 
finement plans it has devised over the current base plan 11 by solving g G 
openGoalsili)^ Refinements’^filg), to the rest of agents in the task. In turn, 
agent i receives the refinements devised by each agent j in the task. Refinements^ 
(Ilg), where j yf i. Note that agents have a local, partial view of the plans, 
so given a refinement plan 11, an agent i will only view the open goals og G 
openGoals{Il) of the form {v,d) or {v,^d) such that u G V* and d G I?*. The 
view agent i has on each refinement plan 11, view'^ill), ensures agents’ privacy 
and directly affects the evaluation of the refinements. 

The evaluation of the refinement plans is carried out through a utility function 
J-, by which agents estimate the quality of the plans. Since an agent i evaluates 
a plan accordingly to its view, iF(view'^(Il)), the results of the evaluation may be 
different from the other agents’. Therefore, agents will have different perspectives 
on the quality of the refinement plans. 

Once the refinement plans are evaluated, agents vote for the most promising 
candidate in i?, which stores all the refinement plans that have not yet been 
selected as a base plan (see Algorithm . Each agent i votes for the best re¬ 
finement plan in R according to the utility function if. In case of a draw, the 
baton agent will choose the next base plan among the most voted alternatives. 
If i? = 0, the refinement planning process ends with no solution found. 

Once a refinement plan is selected, agents adopt it as the new base plan 11. 
If openGoalsfn) = 0, a solution plan is returned. As some open goals might 
not be visible to some agents, all agents must confirm that 11 is a solution plan 
according to their view of 11. Finally, the baton agent selects the next open 
goal g G openGoals(Il) to be solved, and a new iteration of the refinement and 
coordination process starts. 

The resolution process carried out by the agents can be regarded as a joint 
exploration of the refinement space. Nodes in the search tree represent refinement 
plans and each iteration of the process expands a different node. 


9.3. Resolution example 

This subsection illustrates the resolution process by showing a partial trace that 
follows the trace example described in subsection |8.1| After completing the ini¬ 
tial information exchange and building the dis-RPG, agents proceed with the 
resolution process in order to solve the MAP task. 

The plan construction starts with an initial empty plan, Hq, which contains 
only the two fictitious steps that represent the initial state and the goals of the 
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• Ag1 



Fig. 4. Refinement plan IIoo 



Fig. 5. Refinement plan Iloe as observed by: a) Agl b) Ag3 


MAP task. The first open goal selected by Agl (which takes the role of baton 
agent in this first iteration) for its resolution is (= (pos pi) cA), as it is the 
most costly one according to the dis-RPG. The goals of the task are highlighted 
in bold in TableThis dis-RPG shows that (= (pos pi) cA) has a cost of 5, 
(= (pos p3) cF) has a cost of 4, and the only agent that can achieve (= (pos 
pi) cA) is Agl. Hence, Agl proposes a set of refinements over Hq, Hoo, ■. ■, Hog, 
while Ag2 and Ag3 refrain from making proposals. The proposed refinements 
are evaluated through the utility function T, and the best-valued one, Hgo, is 
selected as the new base plan. 

Figure depicts the refinement plan Hgo. Since all the causal links in Hgo 
involve shareable fluents, all the agents have a complete view of this refinement 
plan. However, agents Agl and Ag3 have different views of the refinement Hge 
(see Figure^. In order to guarantee privacy, several causal links (black arrows) 
of Hoe have oeen occluded to Ag3, which only sees ordering constraints instead 
(grey arrows). According to the problem definition files, the fluents involved in 
these causal links are private to the transport agent Agl because they are not 
shareable data, and therefore, Agl does not communicate them to Ag3. 

Once the refinement plan Hgo is chosen as the new base plan, Agl passes on 
the baton to Ag2 and a new iteration of the resolution process starts. The MAP 
process will carry on until a solution plan is found. Since some open goals are 
not visible to some agents, all participating agents must confirm that the plan 
has no pending open goals. Figure [^depicts the solution plan for the MAP task 


• Agl 



Fig. 6. Solution plan for the MAP task 
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at hand, showing in different shapes the actions to be executed by each agent. 
As it can be observed, the solution of the MAP task is a joint plan to which all 
the participant agents have contributed. 


10. Experimental results 

Several tests have been performed to evaluate the performance of our MAP 
system. The tests compare the MAP model with a single-agent approach to an¬ 
alyze its advantages and shortcomings against a centralized planning model. We 
have used two different planning domains for our experiments. Next subsections 
present the MAP domains and analyze the results of the different tests. 


10.1. Multi-agent planning domains 

The two planning domains used to test the MAP system have been taken from 
real-life problems or adapted from well-known case studies. The two domains 
were designed such that we could test the performance and the quality of the 
solutions obtained with a wide range of problems. We tested our MAP system 
with different levels of complexity: from loosely-coupled problems, in which inter¬ 
actions among agents are rather low, to strongly-related problems, that require 
a strong coordination effort to be solved. Additionally, we created both a multi¬ 
agent and a single-agent version for each problem. 

In section we described a transportation and storage domain, in which 
agents take the roles of transport agencies and storage facilities, which work 
together to transport raw materials and final products to different cities. This 
domain gives rise to strongly-related problems as interactions between agents are 
required in order to accomplish the different objectives. Agents in the transporta¬ 
tion domain have different abilities, so they should cooperate with each other in 
order to achieve the different goals. 

We defined an additional planning domain, the picture domain. This domain 
gives rise to simpler, loosely-coupled problems as agents can work independently 
in order to solve the objectives, and hence cooperation and interactions among 
agents are not mandatory to find a solution. Planning agents in the picture 
domain (workers) are not specialized, they all share the same abilities and so 
they all can perform the same actions. In addition, agents in this domain do 
not keep private information for themselves but all the problem information is 
shared among the agents. Next subsection describes the picture domain. 

10.1.1. Picture domain 

This domain, adapted from the case study in (Parsons, Sierra and Jennings, 
1998), presents a situation in which several workers have to cooperate to hang 
a set of pictures on walls. To do so, they have to acquire different tools that are 
scattered over several locations. Agents move through the locations to get the 
tools and hang the pictures. The domain defines a set of bidirectional links that 
connect the locations. 

Figure depicts an example of this planning domain. In contrast to the 
transportation domain, agents in the picture domain share the same capabili¬ 
ties: agents can pickUp and putDown a tool in the location where the agent 
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and the tool are placed; an agent can also pass the tool it is carrying on to 
another agent at the same location; agents can walk from one location to 
another through the link that connects both locations; finally, an agent can hang 
a picture on a certain location with the tool it is carrying. 

This domain gives rise to loosely-coupled problems because an individual 
agent is likely to solve the problem goals by itself in most cases. Moreover, 
agents share the same abilities and have access to all the locations, so they are 
able to work independently and cooperation is not a requirement to complete 
the task. Cooperation is however useful to improve the quality of the solutions 
and to solve conflicts on the use of the tools, as they are limited resources shared 
by all the participating agents. 

10.2. Tests and results 

The following subsections show the experimental results. We carried out two 
different test^ The first one compares the quality of the solution plans obtained 
by a single agent and by a set of planning agents working together on the problem. 
To do so, we defined a set of MAP problems and the single-agent equivalent 
version. Finally, we measured the robustness and scalability of the MAP system 
by executing a planning problem several times, increasing each time the number 
of planning agents in the system. 

10.2.1. Multi-Agent vs. Single-Agent Planning 

This first set of tests compares the quality of the solution plans of our MAP 
approach versus the centralized single-agent framework. The testbed includes 
twenty planning problems (ten problems per domain) of increasing difficulty. 

As stated in subsection |10.1[ the transportation problems present a high cou¬ 
pling level because agents are required to interact between each other to solve 
most of the problem goals. In contrast, agents in the picture problems can solve 


^ All the tests were performed on a single machine with a 2.83 GHz Intel Core 2 Quad CPU 
and 8 GB RAM. 
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• Agi 
■ Ag2 



Fig. 8. Solution plan for the Picture2 MAP problem 


the goals independently in most cases so the coupling level in these problems 
is rather low. Another key difference between both domains is that planning 
agents in the picture domain (workers) are also the entities that execute the 
plans, whereas agents in the transportation domain are merely planning entities. 
This way, given two parallel actions in a plan of the picture domain, each one 
is associated to a different agent (worker) whereas two parallel actions in a plan 
of the transportation domain can be associated to two different trucks of the 
same transport agent, which is the planning entity. In other words, concurrency 
is associated to the agents in the picture domain and to the resources managed 
by the agents (trucks, hoists, etc.) in the transportation domain. 

Table shows the obtained results. #Ag indicates the number of agents that 
perform the planning problem in the MAP tests. Actions and #TS refer to the 
number of actions and time steps of the solution plan, respectively (notice that 
we do not count the plans’ fictitious actions). Finally, Parallelism indicates the 
maximum number of parallel branches in the MAP solution plans. 

Time steps are the number of time units necessary to execute the plan, i.e., 
the duration of the plan. For instance. Figure depicts the solution plan for 
the Picture2 MAP problem. Although the plan is composed of twelve planning 
actions (without taking into account the two fictitious actions), it can be executed 
in only eight time steps, since most of its actions can be executed in two parallel 
branches. Then, the duration of the plan in Figure]^ is 8 time units. 

Discussion on the results. In the transportation domain, the MAP approach 
obtains the same results than the single-agent approach w.r.t. the number of 
actions and time steps. The single-agent approach performs rather well in this 
particular domain, obtaining good-quality solutions, if not optimal, for almost 
all the tested problems. Notice that the single-agent approach features a single 
planning entity that has a full visibility on the planning problem. Despite the 
fact that the participating agents on the MAP tests have an incomplete view of 
the problem, the results show that MAP agents cooperate effectively, obtaining 
plans of the same quality as the single-agent approach, both in terms of number 
of actions and plan duration (time steps). 

In the transportation domain, planning agents have a set of resources at their 
disposal (truck and hoists) to execute the actions of the plan. Since partial-order 
planners allow for parallelism, both the MAP and single-agent plans contain 
parallel actions. Actions in this domain are executed by the trucks and hoists in¬ 
stead of the planning agents themselves. Hence, the number of parallel branches 
and the duration of the solution plans of this domain is only conditioned by 
the number of available resources (trucks and hoists). For this reason, both ap- 
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Problem 


Multi-Agent Planning 


Single-Agent Planning 

#Ag 

^Actions 

#TS Parallelism 

^Actions 

#TS 

Transportation! 

2 

14 

11 

2 

14 

11 

Transportation2 

2 

11 

9 

2 

11 

9 

Transportations 

3 

9 

5 

2 

9 

5 

Transportation4 

3 

11 

6 

2 

11 

6 

Transportation5 

4 

13 

6 

3 

13 

6 

Transportationb 

4 

11 

5 

3 

11 

5 

Transportation? 

5 

10 

8 

2 

10 

8 

Transportations 

5 

15 

9 

3 

15 

9 

Transportation9 

6 

11 

5 

3 

11 

5 

TransportationlO 

6 

17 

10 

3 

17 

10 

Picture! 

2 

11 

6 

2 

14 

14 

Picture2 

2 

12 

8 

2 

11 

11 

Pictures 

3 

6 

2 

3 

8 

8 

Picture4 

3 

11 

7 

2 

11 

11 

Pictures 

4 

8 

2 

4 

11 

11 

Pictures 

4 

10 

6 

2 

10 

10 

Picture? 

5 

8 

5 

2 

8 

8 

Pictures 

5 

10 

2 

5 

14 

14 

PictureQ 

6 

9 

5 

2 

9 

9 

Picture!0 

6 

12 

2 

6 

17 

17 


Table 4. Single-Agent vs. Multi-Agent Planning comparison 


proaches give rise to plans with the same number of actions and time steps. On 
the basis of these results, we can affirm that the quality of the MAP plans is not 
diminished by the limited view and incomplete information of the agents and 
the existence of private information among agents. 

The results of the picture domain present more differences between both ap¬ 
proaches. The single-agent approach obtains sequential plans because the single 
planning agent is also the only execution entity. MAP, however, takes advantage 
of having several planning/execution agents cooperating. MAP enforces cooper¬ 
ation as agents can work together to reach an objective. For instance. Figure 
shows that an agent can pick up a tool and pass it on to another agent. This 
cooperation improves the solution because it prevents the agent from going for 
the tool and retrace its steps, thus reducing the number of actions of the plan. 
Agents also cooperate by proposing different parts of the plan that can be exe¬ 
cuted concurrently, which reduces the duration of the plans with respect to the 
centralized approach. Table shows that all the MAP solution plans for the 
picture domain include at least two parallel branches of actions, meaning that at 
least two agents work concurrently, which improves the quality of the solutions 
as shown in Table lU 

In conclusion, while being a more costly approach (see next subsection for 
scalability tests), MAP obtains equal or better solution plans in terms of both 
number of actions and duration of the plans than the single-agent model. We have 
shown that MAP promotes cooperation among agents thus improving the quality 
of the solution. In addition, MAP agents manage their incomplete information on 
the MAP task efficiently as the quality of the solution plans is not affected, being 
at least on par with the single-agent approach. Moreover, results show that our 
approach obtains good-quality solution plans for problems with different coupling 
and complexity levels, from loosely-coupled to strongly-related problems. 
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Fig. 9. Scalability results for the transportation domain 



Fig. 10. Scalability results for the picture domain 


10.2.2. Scalability analysis 

In this subsection we evaluate the scalability of our MAP framework, i.e., how 
the number of agents in the MAP system affects its efficiency. To do so, eight 
different test problems were generated for both the transportation and the picture 
domains. Each test increases the number of agents by one, keeping the rest of 
the planning problem’s parameters unchanged. 

All the transportation tests include ten different cities, one truck, one empty 
table in the warehouse and one package of raw material. All the problems include 
a single warehouse agent, and each of them adds an extra transport agent, up to 
eight transport agents. The problem goal for all the test problems is to deliver 
the raw material to the warehouse, which must place it on the empty table. The 
optimal solution plan for all the problems includes ten actions and involves the 
participation of at least one transport agent and the warehouse agent. 

As for the picture domain, all the test problems include two different tools and 
twelve different locations. The goal for all the problems is to hang two different 
pictures. The optimal solution plan for these problems has eight actions and 
involves the participation of two different agents. Each agent picks up one tool 
and hangs one picture. 

Figures[^and[TO|depict the results for each domain. As it can be observed, the 
number of messages experiences a notable increase with each new agent included 
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in the MAP process. So does the execution time, which is conditioned by the 
number of messages exchanged among agents. 

Discussion on the results. These results are caused by the growing number of 
refinement plans proposed by the agents. Refinement plans are communicated to 
all the agents in the MAP system, reason why the addition of a planning agent 
represents such an overhead as each new agent proposes and communicates a 
number of extra refinement plans. In addition, the refinement plans proposed by 
each new planning agent increase the complexity of the search tree as they may 
also be adopted as base plans at some point. 

Notice that the number of messages is much larger in the case of the picture 
problems, even though we have defined similar size and complexity problems for 
the two planning domains. This is due to the loosely-coupled nature of the picture 
problems because agents in this domain share the same abilities and every agent 
can make a plan proposal over any base plan. 

As opposite to the picture domain, agents in the transportation domain are 
specialized, which makes them unable to make plan refinements over every base 
plan. Transport agents are limited by their working areas, while warehouse agents 
cannot take part in the transportation of the packages. This fact limits the 
number of exchanged messages, which also benefits the execution time. This 
way, our system proves to be more stable when solving strongly-related problems 
like the transportation tests since the addition of a new agent causes a lower 
increase in the number of messages, which directly affects the execution time. 

In conclusion, the number of agents in the MAP system is a parameter that 
has a significant influence on its efficiency because the number of messages among 
agents constitutes one of the bottlenecks of the system. This issue is more no¬ 
ticeable when dealing with loosely-coupled problems, as agents can devise plan 
proposals over almost any base plan, whereas our MAP system shows a more ro¬ 
bust behavior when solving strongly-related problems. Therefore, our immediate 
challenge is to reduce the number of messages between agents. This way, we will 
improve the scalability of the system and we will be able to test more complex 
planning problems. 

11. Conclusions 

This article presents a MAP model that allows agents to plan under incomplete 
information. Our approach is suitable to solve a wide range of MAP problems, 
from strongly-related problems with a high degree of interaction among agents 
to simpler loosely-coupled problems, which present limited interactions among 
agents. Our model allows for heterogeneous agents with different information, 
capabilities and private goals to cooperatively build a joint plan while handling 
an incomplete view of the MAP task. Agents keep their private data and share 
only the relevant information for their interactions with other agents, thus being 
unaware of part of the information managed by the rest of agents. 

Shareable information is defined through our MAP language, extended from 
PDDL3.1. The information exchange is carried out through the construction of 
a distributed Relaxed Planning Graph, by which agents share the public fluents 
and estimate the best cost to achieve them. 

The MAP resolution process is based on a refinement planning procedure 
whereby agents propose successive refinements to an initially empty base plan 
until a consistent joint plan is obtained. This procedure, that iteratively combines 
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planning and coordination, uses single-agent planning technology to build the 
refinement plans. More precisely, we adapt the POP paradigm to a MAP context, 
which allows agents to build refinement plans leaving details unresolved that will 
be gradually completed by other agents until a solution plan is found. 

Conclusions drawn from the experiments show that MAP agents obtain so¬ 
lution plans of equal or better quality than a single-agent approach for both 
loosely-coupled and strongly-related problems. Despite agents do not have a com¬ 
plete view of the MAP task and keep private information, the quality of the MAP 
solution plans is not affected, neither in terms of number of actions nor plan du¬ 
ration. Hence, we can affirm that our model tackles large MAP tasks in which 
information is distributed among a number of planning entities at least as effec¬ 
tively as a single-agent planning approach working under complete information. 

Moreover, our MAP approach enforces cooperation among agents since they 
work together to solve goals more efficiently. MAP improves plan concurrency as 
agents can solve different goals in parallel, which reduces the duration and the 
number of actions of the solution plans. 
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